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DETAILED ACTION 

Response to Arguments 

1. Applicants' arguments filed, filed 11/16/2009, with respect to the 35 U.S.C. 101 
rejections of claims 1-16 have been fully considered but they are not persuasive. 
Supreme Court precedent and recent Federal Circuit decisions indicate that a statutory 
"process" under 35 U.S.C. 101 must (1) be tied to another statutory category (such as a 
particular apparatus); or (2) transform underlying subject matter (such as an article or 
material) to a different state or thing. Additionally, the use of a particular machine or 
transformation must impose a meaningful limit on the claim's scope and must involve 
more than insignificant "extra solution" activity. While instant claims 1-16 recite a series 
of steps or acts to be performed that are positively tied to another statutory category 
that accomplishes the claimed method steps, the particular machine does not impose a 
meaningful limit on the claims' scope and therefore does not qualify as a statutory 
process. For example, "using digital signal processing circuits including a computer- 
readable memory circuit connected to receive said encoded video sequence and 
configured to process said sequence" does not impose a meaningful limit on the claims' 
scope. This is because use of this particular machine does not involve more than a field 
of use limitation. 

2. The remainder of Applicants' arguments, filed 1 1/1 6/2009, with respect to the 35 
U.S.C. 103 rejections have been fully considered but they are not persuasive. 
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3. Applicants assert that Oh fails to teach "decoding the motion vectors of the 
frame" because Oh only teaches encoding, but not decoding. Examiner respectfully 
disagrees. As stated in pg 10, lines 26-28 of Oh, "All necessary motion vectors. . . are 
detected by the adaptive motion estimator". Further, as stated in pg 17, lines 26-29 of 
Oh, " the principles of the invention can be incorporated in an integrated circuit for 
encoding/decoding video data". Because every principle of the Oh invention is taught to 
be implemented in a decoding environment, Oh teaches "decoding the motion vectors 
of the frame". 

4. Applicants assert that Oh fails to teach "estimating global motion between 
frames" because Oh teaches generating global vectors that relate only to a frame sub- 
section, but not to a whole frame. Examiner respectfully disagrees. Oh teaches using 
groups of macroblocks to determine the global motion estimation between frames. 
There is nothing in applicants' claim language which prevents performing global motion 
estimation in such a manner. Furthermore, as stated in pg 9, lines 13-19 of Oh, 

The global motion estimator is updated with MB motion vectors from the 
past processed pictures by the adaptive motion estimator. The preceding 
pictures motion vectors are used to generate one or more global motion vectors 
for each group of MBs in a new picture to be coded based on the type of global 
motion estimator selected. Generated global motion vectors are used to offset 



Application/Control Number: 10/535,621 Page 4 

Art Unit: 2621 

search windows of all MBs in the corresponding group of MBs. For the case of 
more than one global motion vector, a comparison at MB level is done and the 
global motion vector that gives the best result is chosen. 

Thus Oh teaches global motion estimation between a preceding picture and a 
new picture. 

5. Applicants assert that Oh fails to teach "calculation of any error value for motion 
estimation for sets of motion vectors" because Oh only describes use of error values in 
the matching of blocks or macroblocks. Examiner respectfully disagrees. The matching 
of macroblocks between two frames is the process of motion estimation. As 
correspondingly stated in pg 10, line 26-29 of Oh, "All necessary motion vectors, for 
example the frame and/or field, forward and/or backward, and 16x16/16x8/8x8 motion 
vectors, for each MB are detected by the adaptive motion estimator 102 by matching 
the MB to candidate blocks obtained from one or more search windows from a 
reference picture stored in a frame buffer 103". Further, as stated in the same 
paragraph, pg 1 1 , line 4-6 of Oh, "The matching criterion may be based on minimum of 
absolute errors, square errors, or other suitable distortion functions". Since the 
matching of blocks between frames is based on error, and this same matching of blocks 
between frames is defined to be motion estimation, Oh teaches motion estimation using 
an error value. 
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6. Applicants assert that Oh fails to teach "selection of any motion estimation for 
sets of motion vectors as representative of the global motion of the frame with respect 
to a preceding or succeeding anchor frame" because Oh only selects a prediction of the 
position of a macroblock in a subsequent image that best matches a block in the current 
image, and Oh does not select a motion estimation for sets of motion vectors. 
Examiner respectfully disagrees. As shown above, the matching of macroblocks 
between two frames is the process of motion estimation. Additionally, as stated in pg 
14, line 24-28 of Oh, "The two best macroblock matching predictions obtained from the 
two search windows are compared and the one giving the best prediction is chosen , 
with the corresponding MB motion vector assigned to the current MB. If only one global 
motion vector is used (depending on the type of motion estimator chosen), then the 
procedure illustrated previously for the first search window is carried out." Thus Oh 
teaches selection of a motion estimation for global estimation from two possible motion 
estimations. 

7. Applicants assert that Oh in view of Meer fails to teach "calculating a median 
squared error value for each motion estimation". Examiner respectfully disagrees. As 
shown above, Oh teaches motion estimation using an error value. While Oh teaches 
using "absolute errors, square errors, or other suitable distortion functions" for each 
motion estimation, as stated in pg 11, line 4-6 of Oh, Oh fails to specifically teach using 
a least median squared error value. A least median squared error value is taught by 
Meer on pg 62, right column. Using such a least median squared error value as taught 
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by Meer would improve the Oh device to identify "outliers" which are points severely 
deviating from the model, and discriminate against these points, thereby optimizing the 
ability of Oh to generate accurate global motion in the presence outliers. Thus Oh in 
view of Meer teaches using a median square error value for each motion estimation. 

Claim Rejections - 35 USC § 101 

8. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

9. Claims 1-16 are rejected under 35 U.S.C. 101 as not falling within one of the four 
statutory categories of invention. Supreme Court precedent and recent Federal Circuit 
decisions indicate that a statutory "process" under 35 U.S.C. 101 must (1) be tied to 
another statutory category (such as a particular apparatus); or (2) transform underlying 
subject matter (such as an article or material) to a different state or thing. Additionally, 
the use of a particular machine or transformation must impose a meaningful limit on the 
claim's scope and must involve more than insignificant "extra solution" activity. While 
instant claims 1-16 recite a series of steps or acts to be performed that are positively 
tied to another statutory category that accomplishes the claimed method steps, the 
particular machine does not impose a meaningful limit on the claims' scope and 
therefore does not qualify as a statutory process. For example, "using digital signal 
processing circuits including a computer-readable memory circuit connected to receive 
said encoded video sequence and configured to process said sequence" does not 
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impose a meaningful limit on the claims' scope. This is because use of this particular 
machine does not involve more than a field of use limitation. 

Claim Rejections - 35 USC § 103 

1 0. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

1 1 . Claims 1,5,17,21 and 32 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Oh et al. (International Pub. No. WO 00/70879) in view of Meer et al. 
(Meer et al., "Robust Regression Methods for Computer Vision: A Review", International 
Journal of Computer Vision, 6(1):59-70, 1991). 

12. Regarding claim 1, 
Oh teaches: 

A method of global motion estimation between frames of a motion-compensated 
inter-frame encoded video sequence, each frame of the sequence having a plurality of 
motion vectors encoded therein relating the frame to a preceding and/or succeeding 
frame of the sequence; the method comprising using digital signal processing circuits 
including a computer-readable memory circuit connected to receive said encoded video 
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sequence and configured to process said sequence, for a particular inter-frame 
encoded frame: 

a) decoding the motion vectors of the frame (Oh: pg 10, lines 26-28: All 
necessary motion vectors. . . are detected by the adaptive motion estimator; pg 17, lines 
26-29: the principles of the invention can be incorporated in an integrated circuit for 
encoding/decoding video data;); 

b) selecting N sets of the motion vectors, wherein N is an integer value greater 
than 1 (Oh: pg 12, lines 24-30: In this particular embodiment, three global motion 
vectors are computed: one representing the average motion vector of the whole row, 
the other two global motion vectors representing two sub-groups of MBs from the row 
differentiated by some motion characteristics; pg 13, lines 4-7: separate the MB motion 
vectors into two groups, Groupl and Group2;); 

c) calculating a motion estimation for each set (Oh: pg 12, line 19-22: The global 
motion estimator 203 is coupled to provide data to the motion characteristics analyser 
and the motion estimators 204, 205, 206, 207;); 

d) calculating a error value for each motion estimation (Oh: pg 1 1 , line 4-6; The 
matching criterion may be based on minimum of absolute errors, square errors, or other 
suitable distortion functions;); and 

e) selecting the motion estimation as that representative of the global motion of 
the frame with respect to a preceding or succeeding anchor frame (Oh: pg 14, line 24- 
26: The two best macroblock matching predictions obtained from the two search 
windows are compared and the one giving the best prediction is chosen, with the 
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corresponding MB motion vector assigned to the current MB;). 
Oh fails to teach: 

d) calculating a median squared error value for each motion estimation; and 

e) selecting the motion estimation with the least median squared error value 

as that representative of the global motion of the frame with respect to a preceding or 
succeeding anchor frame. 

Meer teaches: 

d) calculating a median squared error value for each motion estimation (Meer: 
pg 62, right col); and 

e) selecting the motion estimation with the least median squared error value 

as that representative of the global motion of the frame with respect to a preceding or 
succeeding anchor frame (Meer: pg 62, right col). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Meer with Oh. The teachings of Meer both 
identifies "outliers", that is points severely deviating from the model, and discriminates 
against these points (Meer: pg 62, right col), thereby optimizing the ability of Oh to 
generate accurate global motion in the presence outliers. 



13. Regarding claim 5, 
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Meer teaches: 

A method according to claim 1, wherein the selecting step b) further comprises 
randomly selecting s motion vectors from the available motion vectors for each of the N 
sets (Meer: pg 64, left col: Subsets of the data are chosen by random sampling) 
wherein s is the minimum number for sufficiently estimating a geometrical 
transformation (Meer: pg 64, left col: The points within some error tolerance are called 
the consensus set of the model. If the cardinality, number of points, of the consensus 
set exceeds a threshold, the model is accepted and its parameters recomputed based 
on the whole consensus set. By using a threshold, a minimum number of motion 
vectors must be used. The error tolerance and the consensus set acceptance threshold 
must be set a priori, and can be set according to any minimum level, including the 
minimum number for sufficiently estimating a geometrical transformation.). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Meer with Oh. The teachings of Meer choose 
subsets of data by random sampling, thereby minimizing the cost function of the Oh 
method and apparatus (Meer: pg 64, left col). 

14. Regarding claim 1 7, Oh in view of Meer discloses the system limitations of this 
claim as discussed above with respect to claim 1 . 
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1 5. Regarding claim 21 , Oh in view of Meer discloses the system limitations of this 
claim as discussed above with respect to claim 5. 

16. Regarding claim 32, 
Oh teaches: 

A computer-readable storage medium containing a computer program or suite of 
programs arranged such that when executed on a computer system the program or 
suite of programs causes the computer system to perform the method of claim 1 (Oh: 
pg 17, line 29 - pg 18, line 1). 

1 7. Claims 2-4 and 1 8-20 are rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over Oh et al. (International Pub. No. WO 00/70879) in view of Meer et al. (Meer et al., 
"Robust Regression Methods for Computer Vision: A Review", International Journal of 
Computer Vision, 6(1 ):59-70, 1 991 ) as applied to claim 1 above, and further in view of 
Smolic et al. (Smolic et al., "Long-Term Global Motion Estimation and its Application for 
Sprite Coding, Content Description, and Segmentation", IEEE TRANSACTIONS ON 
CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, Vol. 9, No. 8, December 
1999, pages 1227-1242, XP0009333894). 

18. Regarding claim 2, 
Oh in view of Meer teaches: 

A method according to claim 1 (as shown above) 
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Oh in view of Meer fails to teach: 

the step of excluding certain motion vectors with predetermined characteristics 
from being selected as a member of one of the N sets. 

Smolic teaches: 

the step of excluding certain motion vectors with predetermined characteristics 
from being selected as a member of one of the N sets (Smolic: pg 1 231 , right col, first 
paragraph). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Smolic with Oh in view of Meer. Excluding 
motions vectors would reduce the computational complexity of the Oh in view of Meer 
method and apparatus (Smolic: pg 1231, right col, first paragraph). 

19. Regarding claim 3, 
Smolic teaches: 

wherein the excluded motion vectors include those motion vectors from one or 
more areas substantially around the boundary of the frame (Smolic: pg 1 231 , right col, 
first paragraph: we exclude some points along the image border and along the object 
border.). 
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20. Regarding claim 4, 
Smolic teaches: 

wherein the excluded motion vectors include those motion vectors whose value 
is substantially zero (Smolic: pg 1231, right col, first paragraph: we evaluate only points 
where the spatial intensity derivatives have nonzero values). 

21 . Regarding claim 18, Oh in view of Meer and further in view of Smolic discloses 
the system limitations of this claim as discussed above with respect to claim 2. 

22. Regarding claim 1 9, Oh in view of Meer and further in view of Smolic discloses 
the system limitations of this claim as discussed above with respect to claim 3. 

23. Regarding claim 20, Oh in view of Meer and further in view of Smolic discloses 
the system limitations of this claim as discussed above with respect to claim 4. 

24. Claims 6-8 and 22-24 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Oh et al. (International Pub. No. WO 00/70879) in view of Meer et al. (Meer et al., 
"Robust Regression Methods for Computer Vision: A Review", International Journal of 
Computer Vision, 6(1 ):59-70, 1 991 ) as applied to claim 1 above, and further in view of 
Subramaniyan et al. (US Patent Application Publication # 2004/0028134 A1). 



25. Regarding claim 6, 
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Oh in view of Meer teaches: 

A method according to claim 1 (as shown above) 

Oh in view of Meer fails to teach: 

f) testing the selected motion estimation representative of the global motion ; and 

g) outputting the selected motion estimation as being representative of the global 
motion of the frame if the test is passed. 

Subramaniyan teaches: 
the steps of: 

f) testing the selected motion estimation representative of the global motion 
(Subramaniyan: [0035], comparing the difference metric for each of the final motion 
vectors in the previous frame with a predetermined threshold; [0049], This process 
continues until one of 3 conditions are met: [0050], CONDITION 1 : The MSAD is below 
a threshold THRESH4 , given by: THRESH4=A*Q+B); and 

g) outputting the selected motion estimation as being representative of the global 
motion of the frame if the test is passed (Subramaniyan: [0035], determining the global 
motion vector based on the each of the final motion vectors in a previous frame with a 
difference metric that is below the threshold; [0049], This process continues until one of 
3 conditions are met: [0050], CONDITION 1 : The MSAD is below a threshold 
THRESH4 . given by: THRESH4=A*Q+B). 
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At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Subramaniyan with Oh in view of Meer. 
Testing the global motion estimation before outputting would improve the Oh in view of 
Meer method and apparatus by achieving a motion estimation that is high quality while 
remaining low in complexity (Subramaniyan: [0023]). 

26. Regarding claim 7, 
Subramaniyan teaches: 

wherein the test comprises comparing the selected motion estimation with a 
threshold value (Subramaniyan: [0049]-[0050], THRESH4), wherein the test is passed if 
the parameters do not exceed the threshold value (Subramaniyan: [0049]-[0050], [0054] 
The best MV at the end of the second stage is chosen as the best MV for the 
macroblock; the MV having a MSAD below THRESH4 is chosen as the best MV). 

27. Regarding claim 8, 
Subramaniyan teaches: 

wherein if the test is failed, the method further comprises: 

h) determining a motion estimation representative of the global motion of the 

frame with respect to a preceding or succeeding other frame (Subramaniyan: [0035]; 

[0026], motion estimation is computed for blocks of image data from a current image 

frame using one or more previously processed image frames ): 
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i) determining a motion estimation representative of the global motion of the other 
frame with respect to the anchor frame ([0035]; the motion estimation circuit 1 10 can 
determine the global motion vector by using an average of all final motion vectors in a 
previous frame ): and 

j) accumulating the motion estimations to give an overall motion estimation 
substantially representative of the global motion of the frame with respect to the anchor 
frame (Subramaniyan: [0035], the motion estimation circuit 110 can determine the 
global motion vector by using an average of all final motion vectors in a previous frame). 

28. Regarding claim 22, Oh in view of Meer and further in view of Subramaniyan 
discloses the system limitations of this claim as discussed above with respect to claim 
6. 

29. Regarding claim 23, Oh in view of Meer and further in view of Subramaniyan 
discloses the system limitations of this claim as discussed above with respect to claim 
7. 

30. Regarding claim 24, Oh in view of Meer and further in view of Subramaniyan 
discloses the system limitations of this claim as discussed above with respect to claim 
8. 
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31 . Claims 10 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Oh et al. (International Pub. No. WO 00/70879) in view of Meer et al. (Meer et al., 
"Robust Regression Methods for Computer Vision: A Review", International Journal of 
Computer Vision, 6(1):59-70, 1991) and Subramaniyan et al. (US Patent Application 
Publication # 2004/0028134 A1 ) as applied to claim 6 above, and further in view of Lee 
et al. (US Patent Application Publication # 2003/0103568 A1). 

32. Regarding claim 10, 

Oh in view of Meer and Subramaniyan teaches: 

A method according to claim 6 (as shown above) 

Oh in view of Meer and Subramaniyan fails to teach: 

wherein if the test is failed, the method further comprises: 

interpolating between the motion estimations of adjacent frames to give an 

interpolated motion estimation which is then output as the motion estimation 

representative of the global motion of the frame. 

Lee teaches: 

wherein if the test is failed, the method further comprises: 
interpolating between the motion estimations of adjacent frames to give an 
interpolated motion estimation which is then output as the motion estimation 
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representative of the global motion of the frame (Lee: [0061]-[0063]). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Lee with Oh in view of Meer and 
Subramaniyan. The teachings of Lee provide for motion compensated interpolation that 
eliminates blocking artifacts (Lee [0048]), thereby increasing the ability of Oh in view of 
Meer and Subramaniyan to generate accurate global motion in the presence of blocking 
artifacts. 

33. Regarding claim 25, Oh in view of Meer and Subramaniyan and further in view of 
Lee discloses the system limitations of this claim as discussed above with respect to 
claim 10. 

34. Claims 26-29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Oh et al. (International Pub. No. WO 00/70879) in view of Meer et al. (Meer et al., 
"Robust Regression Methods for Computer Vision: A Review", International Journal of 
Computer Vision, 6(1 ):59-70, 1 991 ) as applied to claim 1 7 above, and further in view of 
Jinzenji et al. (US Patent # 6,977,664 B1 ). 



35. Regarding claim 26, 
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Oh in view of Meer teaches: 

A system for generating panoramic images from a motion-compensated inter-frame 
encoded video sequence, comprising: 

a system for global motion estimation between frames of a motion-compensated 
inter-frame encoded video sequence as claimed in claim 17 (as shown above), and 
further arranged to provide global motion estimations for each frame (Oh: pg 9, lines 13- 
19); 

Oh in view of Meer fails to teach: 

panoramic image generating means for generating at least one panoramic image 
representing the frames of the video sequence using the global motion estimations thus 
determined. 

Jinzenji teaches: 

panoramic image generating means for generating at least one panoramic image 
representing the frames of the video sequence using the global motion estimations thus 
determined (Jinzenji col 8, line 32-56; a provisional sprite <panoramic image> is 
generated). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Jinzenji with Oh in view of Meer. Using the 
panoramic generating means of Jinzenji would allow for creation of a panoramic image 
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based on the global motion of Oh in view of Meer, thereby allowing a user to implement 
an established use for global motion. 

36. Regarding claim 27, 
Jinzenji teaches: 

wherein the panoramic image generating means is further arranged in use to: 

select a particular frame of the sequence as a reference frame, the plane of the 
reference frame thereby being a reference plane (Jinzenji col 8, line 32-56; the 
reference coordinate system which is for the reference frame); 

for each frame other than the reference frame, accumulate the global motion 
estimations from each frame back to the reference frame (Jinzenji col 8, line 32-56; 
each original image of the arbitrary frames is mapped to a reference coordinate system 
which is for the reference frame); 

warp each frame other than the reference frame onto the reference plane using 
the accumulated global motion estimations to give one or more pixel values for each 
pixel in the reference plane (Jinzenji col 8, line 32-56; frame warping occurs when 
frames are mapped to the reference coordinate system using global motion so as to 
insert or overwrite pixels); and 

for each pixel position in the reference plane, select one of the available pixel 
values for use as the pixel value in the panoramic image (Jinzenji col 8, line 32-56; a 
pixel value of a point is obtained from pixel values which exist in the same point). 
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37. Regarding claim 28, 
Jinzenji teaches: 

wherein the panoramic image generating means is further arranged to select a 
substantially median pixel value from the available pixel values for use in a background 
panoramic image (Jinzenji col 1 0, line 8-1 1 ; for a plurality of pixels which are mapped to 
the same coordinates, a median value of the pixels is selected as the value of the 
coordinates of the provisional sprite). 

38. Regarding claim 29, 
Jinzenji teaches: 

wherein the panoramic image generating means is further arranged to select a 
substantially most different pixel value from the available pixel values for use in a 
foreground panoramic image (Jinzenji col 8, line 47-51; using a threshold to select the 
most different pixel). 

39. Claims 30-31 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Oh et al. (International Pub. No. WO 00/70879) in view of Meer et al. (Meer et al., 
"Robust Regression Methods for Computer Vision: A Review", International Journal of 
Computer Vision, 6(1 ):59-70, 1 991 ) and Jinzenji et al. (US Patent # 6,977,664 B1 ) as 
applied to claim 27 above, and further in view of Szeliski et al. (US Patent # 6,348,91 8 
B1). 
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40. Regarding claim 30 and 31 , 

Oh in view of Meer and Jinzenji teaches: 

A system according to claim 26 (as shown above), 

Oh in view of Meer and Jinzenji fails to teach: 

wherein the panoramic image generating means is further arranged to: 

calculate the mean pixel value of the available pixel values; 

calculate the L1 distance between each available pixel value and the calculated 
mean pixel value; and 

select the pixel value with the median L1 distance for use in a background 
panoramic image. 

select the pixel value with the maximum L1 distance for use in a foreground 
panoramic image. 

Szeliski teaches: 

wherein the panoramic image generating means is further arranged to: 

calculate the mean pixel value of the available pixel values (Szeliski col 8, line 

57-65; taking the mean of the color or intensity values); 

calculate the L1 distance between each available pixel value and the calculated 

mean pixel value (Szeliski col 8, line 57-65; where the averaging is weighted by the 

distance of each pixel from the nearest invisible pixel); and 
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select the pixel value with the median L1 distance for use in a background 
panoramic image (Szeliski col 8, line 57-65; using the median technique). 

select the pixel value with the maximum L1 distance for use in a foreground 
panoramic image (Szeliski col 8, line 57-65; the simplest technique is the median 
technique, but many others exist. This portion of Szeliski discloses blending specifically 
for a background image. This portion of Szeliski also discloses blending generally. 
Instead of using the median technique for blending background pixels, a maximum 
technique is obvious for blending foreground pixels. This is because foreground pixels 
are most different from background pixels). 

It would have been obvious to a person having ordinary skill in the art to combine 
the teachings of Szeliski with Oh in view of Meer and Jinzenji. Using the blending 
technique of Szeliski would smooth out disparities of the panoramic image of Oh in view 
of Meer and Jinzenji, thus creating a panoramic image with increased image quality 
(Szeliski col 9, line 6-8). 

41 . Claims 11-14 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Oh et al. (International Pub. No. WO 00/70879) in view of Meer et al. (Meer et al., 
"Robust Regression Methods for Computer Vision: A Review", International Journal of 
Computer Vision, 6(1 ):59-70, 1 991 ), and further in view of Jinzenji et al. (US Patent # 
6,977,664 B1). 
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42. Regarding claim 11, 

Oh teaches: 

A method of generating panoramic images from a motion-compensated inter- 
frame encoded video sequence, the method comprising using digital signal processing 
circuits including a computer-readable memory circuit connected to receive said 
encoded video sequence and configured to process said sequence by: 

for each frame of the sequence, determining the global motion of each frame with 
respect to its anchor frame by estimating global motion between frames of a motion- 
compensated inter-frame encoded video sequence (Oh: pg 9, lines 13-19): 

a) decoding the motion vectors of the frame (Oh: pg 10, lines 26-28: All 
necessary motion vectors. . . are detected by the adaptive motion estimator; pg 17, lines 
26-29: the principles of the invention can be incorporated in an integrated circuit for 
encoding/decoding video data;); 

b) selecting N sets of the motion vectors, wherein N is an integer value greater 
than 1 (Oh: pg 12, lines 24-30: In this particular embodiment, three global motion 
vectors are computed: one representing the average motion vector of the whole row, 
the other two global motion vectors representing two sub-groups of MBs from the row 
differentiated by some motion characteristics; pg 13, lines 4-7: separate the MB motion 
vectors into two groups, Groupl and Group2;); 

c) calculating a motion estimation for each set (Oh: pg 12, line 19-22: The global 
motion estimator 203 is coupled to provide data to the motion characteristics analyser 
and the motion estimators 204, 205, 206, 207;); 
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d) calculating a error value for each motion estimation (Oh: pg 1 1 , line 4-6; The 
matching criterion may be based on minimum of absolute errors, square errors, or other 
suitable distortion functions;); and 

e) selecting the motion estimation as that representative of the global motion of 
the frame with respect to a preceding or succeeding anchor frame (Oh: pg 14, line 24- 
26: The two best macroblock matching predictions obtained from the two search 
windows are compared and the one giving the best prediction is chosen, with the 
corresponding MB motion vector assigned to the current MB;). 

Oh fails to teach: 

d) calculating a median squared error value for each motion estimation; and 

e) selecting the motion estimation with the least median squared error value 

as that representative of the global motion of the frame with respect to a preceding or 
succeeding anchor frame. 

f) generating at least one panoramic image representing the frames of the video 
sequence using the global motion estimations thus determined. 

Meer teaches: 

d) calculating a median squared error value for each motion estimation (Meer: 
pg 62, right col); and 
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e) selecting the motion estimation with the least median squared error value 

as that representative of the global motion of the frame with respect to a preceding or 
succeeding anchor frame (Meer: pg 62, right col). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Meer with Oh. The teachings of Meer both 
identifies "outliers", that is points severely deviating from the model, and discriminates 
against these points (Meer: pg 62, right col), thereby optimizing the ability of Oh to 
generate accurate global motion in the presence outliers. 

Jinzenji teaches: 

f) generating at least one panoramic image representing the frames of the video 
sequence using the global motion estimations thus determined (Jinzenji col 8, line 32- 
56; a provisional sprite <panoramic image> is generated). 

At the time of invention, it would have been obvious to a person having ordinary 
skill in the art to combine the teachings of Jinzenji with Oh in view of Meer. Using the 
panoramic generating means of Jinzenji would allow for creation of a panoramic image 
based on the global motion of Oh in view of Meer, thereby allowing a user to implement 
an established use for global motion. 
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43. Regarding claim 12, Jinzenji teaches: 

A method according to claim 1 1 , wherein the generating step further comprises: 

selecting a particular frame of the sequence as a reference frame, the plane of 
the reference frame being a reference plane (Jinzenji col 8, line 32-56; the reference 
coordinate system which is for the reference frame); 

for each frame other than the reference frame, accumulating the global motion 
estimations from each frame back to the reference frame (Jinzenji col 8, line 32-56; 
each original image of the arbitrary frames is mapped to a reference coordinate system 
which is for the reference frame); 

warping each frame other than the reference frame onto the reference plane 
using the accumulated global motion estimations to give one or more pixel values for 
each pixel position in the reference plane (Jinzenji col 8, line 32-56; frame warping 
occurs when frames are mapped to the reference coordinate system using global 
motion so as to insert or overwrite pixels; and 

for each pixel position in the reference plane, selecting one of the available pixel 
values for use as the pixel value in the panoramic image (Jinzenji col 8, line 32-56; a 
pixel value of a point is obtained from pixel values which exist in the same point). 

44. Regarding claim 13, Jinzenji teaches: 

A method according to claim 12, wherein the selecting step comprises selecting 
a substantially median pixel value from the available pixel values for use in a 
background panoramic image (Jinzenji col 10, line 8-1 1 ; for a plurality of pixels which 
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are mapped to the same coordinates, a median value of the pixels is selected as the 
value of the coordinates of the provisional sprite). 

45. Regarding claim 14, Jinzenji teaches: 

A method according to claim 12, wherein the selecting step comprises selecting 
a substantially most different pixel value from the available pixel values for use in a 
foreground panoramic image (Jinzenji col 8, line 47-51; using a threshold to select the 
most different pixel). 

46. Claims 1 5-1 6 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Oh et al. (International Pub. No. WO 00/70879) in view of Meer et al. (Meer et al., 
"Robust Regression Methods for Computer Vision: A Review", International Journal of 
Computer Vision, 6(1 ):59-70, 1 991 ) and Jinzenji et al. (US Patent # 6,977,664 B1 ) as 
applied to claim 1 1 above, and further in view of Szeliski et al. (US Patent # 6,348,91 8 
B1). 

47. Regarding claims 15-16, 

Oh in view of Meer and Jinzenji teaches: 

A method according to claim 12 (as shown above), 

Oh in view of Meer and Jinzenji fails to teach: 
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wherein the selecting step comprises: 

calculating the mean pixel value of the available pixel values; 

calculating the L1 distance between each available pixel value and the calculated 
mean pixel value; and 

select the pixel value with the median L1 distance for use in a background 
panoramic image. 

select the pixel value with the maximum L1 distance for use in a foreground 
panoramic image. 

Szeliski teaches: 

wherein the selecting step comprises: 

calculating the mean pixel value of the available pixel values (Szeliski: col 8, line 
57-65; taking the mean of the color or intensity values); 

calculating the L1 distance between each available pixel value and the calculated 
mean pixel value (Szeliski: col 8, line 57-65; where the averaging is weighted by the 
distance of each pixel from the nearest invisible pixel); and 

select the pixel value with the median L1 distance for use in a background 
panoramic image (Szeliski: col 8, line 57-65; using the median technique). 

select the pixel value with the maximum L1 distance for use in a foreground 
panoramic image (Szeliski: col 8, line 57-65; the simplest technique is the median 
technique, but many others exist. This portion of Szeliski discloses blending specifically 
for a background image. This portion of Szeliski also discloses blending generally. 
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Instead of using the median technique for blending background pixels, a maximum 
technique is obvious for blending foreground pixels. This is because foreground pixels 
are most different from background pixels). 

It would have been obvious to a person having ordinary skill in the art to combine 
the teachings of Szeliski with Oh in view of Meer and Jinzenji. Using the blending 
technique of Szeliski would smooth out disparities of the panoramic image of Oh in view 
of Meer and Jinzenji, thus creating a panoramic image with increased image quality 
(Szeliski col 9, line 6-8). 

Conclusion 

48. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

49. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James Pontius whose telephone number is (571) 270- 
7687. The examiner can normally be reached on Monday - Thursday, 8 AM - 4 PM est.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/James Pontius/ 
Examiner, Art Unit 2621 



Application/Control Number: 10/535,621 Page 32 

Art Unit: 2621 

/Dave Czekaj/ 

Primary Examiner, Art Unit 2621 



